fix(passthrough): stop leaking the caller's virtual key on credential-less Vertex passthrough - #38114
Merged
mateo-berri merged 12 commits intoAug 24, 2026
Conversation
…-less Vertex passthrough When no Vertex credential is configured (no default_vertex_config, no matching use_in_pass_through deployment, no vector-store credential), the Vertex passthrough took the bring-your-own-credentials branch and forwarded the entire incoming header set upstream to Google. That set included whichever header carried the caller's LiteLLM virtual key: x-litellm-api-key, or Authorization when get_litellm_virtual_key read the key from there. The proxy's own secret was sent to a third-party provider. The credential-less branch now drops x-litellm-api-key and the Authorization value that equals the virtual key, keeping a genuine bring-your-own Google credential (an OAuth token in Authorization, or x-goog-api-key) so real BYO passthrough still works. When neither survives, the request fails with a clean 401 telling the operator no credential is configured, instead of forwarding the virtual key. Regression coverage in the mapped test path asserts the 401-and-never-forwarded behavior for both leak vectors and that a real Google credential still passes through with the virtual key stripped.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis PR hardens credential-less Vertex passthrough handling and rejects requests that lack a usable upstream credential
Confidence Score: 5/5The PR appears safe to merge No blocking failure remains
|
| Filename | Overview |
|---|---|
| litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py | Adds credential-aware filtering and an early rejection path; the fixes address all displayed prior findings |
| tests/test_litellm/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py | Adds regression tests for credential source precedence, configured headers, supported schemes, and bring-your-own credentials |
Reviews (11): Last reviewed commit: "fix(vertex-passthrough): cover the mappe..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…ss Vertex forward The credential-less Vertex passthrough dropped the caller's LiteLLM virtual key only from Authorization by exact match. A caller who sent the same key in x-goog-api-key (which doubles as a real Google credential) had it accepted as a credential and forwarded upstream. Drop the virtual key by value across every forwarded header, normalizing any Bearer prefix, so no header name carries it to Google.
Contributor
Author
Adds a regression asserting the value-based strip also drops the caller's virtual key when it is duplicated into the api-key and x-api-key headers, while a genuine bring-your-own Google credential still forwards.
On the credential-less Vertex passthrough branch, drop every header that can only carry LiteLLM caller auth (x-litellm-api-key, api-key, x-api-key) by name, since Google never consumes them, and strip the virtual key by value from Authorization / x-goog-api-key, which may instead hold a genuine bring-your-own Google credential. This closes the residual leak where a distinct caller secret in api-key or x-api-key still reached upstream.
Contributor
Author
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
user_api_key_auth also authenticates a caller from the operator-configured general_settings.litellm_key_header_name, reading that header straight off the request, so a virtual key sent there survived the credential-less Vertex forwarding filter and reached Google alongside a real bring-your-own credential. Value-strip every header whose value matches the caller's key from any accepted source, including that custom header.
Contributor
Author
…alHeaders The hand-rolled drop set missed Ocp-Apim-Subscription-Key, so a caller Azure APIM secret in that header was forwarded to Google on the credential-less branch. Derive the name-drop set from the canonical SpecialHeaders.litellm_credential_header_names(), minus Authorization and x-goog-api-key which double as real Google credentials and are value-stripped instead. New credential headers added there are now dropped automatically.
Contributor
Author
The credential-less filter derived the caller key only from x-litellm-api-key, Authorization, and the custom header, but the route authenticates through Depends(user_api_key_auth), which also accepts the key from x-goog-api-key. A virtual key sent only in x-goog-api-key therefore authenticated yet was kept as a preserved upstream header and forwarded to Google. Resolve the caller key by the same precedence get_api_key uses and value-strip exactly that, so a key in x-goog-api-key is stripped while a real Google key alongside a higher-precedence virtual key is preserved.
Contributor
Author
…er_token The filter's own Bearer-only stripping missed the other schemes user_api_key_auth accepts, so a virtual key echoed as `Authorization: Basic <key>` alongside a higher-precedence auth header did not match the caller key and was forwarded to Google. Reuse the auth module's _get_bearer_token so the comparison strips exactly what authentication does (Bearer / bearer / Basic / AWS4-HMAC-SHA256), falling back to the raw value for a bare token.
Contributor
Author
…in streaming tests The LIT-4761 streaming-classification tests passed only the bring-your-own Google OAuth token in Authorization and mocked get_litellm_virtual_key, a shape that cannot authenticate in production. The credential-less filter now resolves the caller key by auth precedence, so a lone Authorization value reads as the key and is stripped. Send the virtual key in x-litellm-api-key, matching a real request, so Authorization is preserved and the classification assertions run.
Contributor
Author
…eaders user_api_key_auth also accepts the caller key from a pass_through_endpoints entry's headers.litellm_user_api_key, not just litellm_key_header_name. Drop every operator-configured caller-key header by name and treat them as top-precedence caller-key sources, so a virtual key sent through one is never forwarded to Google.
Contributor
Author
…key headers The resolver placed both operator-configured key headers at the top of its precedence, but user_api_key_auth only overrides with litellm_key_header_name; a pass_through_endpoints litellm_user_api_key is checked last. So a request that authenticated via Authorization while also sending a pass-through header could have the wrong value chosen, leaving the authenticated Authorization key forwarded. Order the resolver exactly like get_api_key: override first, built-in headers next, pass-through header last.
Contributor
Author
Contributor
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…header On mapped pass-through routes, of which /vertex_ai is one, user_api_key_auth accepts the caller key from a header literally named litellm_user_api_key and applies it last, so it overrides every other source. The credential-less filter neither dropped it nor resolved the caller key from it, so a virtual key there reached Google past a real x-goog-api-key, and a bring-your-own Authorization could be stripped when auth actually came from that header. Drop it by name and resolve it at highest precedence.
Contributor
Author
|
bugbot run |
Contributor
Author
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 16a81c9. Configure here.
mateo-berri
enabled auto-merge
August 24, 2026 21:47
7 tasks
tin-berri
approved these changes
Aug 24, 2026
tin-berri
left a comment
tin-berri
left a comment
Contributor
There was a problem hiding this comment.
Approving — real leak, right fix. Forwarding _safe_get_request_headers(request).copy() wholesale meant the caller's virtual key went to Google on every credential-less Vertex passthrough, and splitting it into "drop the proxy-only auth headers by name, drop the value that actually authenticated wherever it appears, 401 if nothing usable survives" is the correct decomposition. Reusing _get_bearer_token from user_api_key_auth rather than re-deriving the scheme stripping is exactly right — that comparison has to agree with authentication or it strips the wrong thing.
Traced the cases that decide whether this is correct or just looks correct:
- virtual key in
Authorizationonly → stripped by value, nothing survives, 401 with the actionable message. Right. - virtual key in
x-litellm-api-key+ real Google token inAuthorization→ key dropped by name,Authorizationvalue ≠ caller key so it's preserved. Right. - virtual key in
x-goog-api-key→ resolved as the authenticated value and dropped by value, not kept just because Google consumes that header. This is the case a by-name-only fix would have missed.
Also checked the dict → Mapping return-type change for runtime breakage: create_pass_through_route does dict(param_custom_headers) at the pass_through_request call, and forward_headers_from_request rebinds ({**request_headers, **headers}) rather than mutating in place, so the MappingProxyType never reaches anything that writes to it.
One thing worth a guard. The whole fix rests on forward_headers being False on this route, and nothing here says so. If it's ever True, forward_headers_from_request merges the raw incoming headers back in — and it only pops a request header when that name is already present in headers, so every header this PR dropped by name (x-litellm-api-key, api-key, x-api-key, Ocp-Apim-Subscription-Key, litellm_user_api_key, the operator-configured ones) gets re-added verbatim and the leak is back. Today _base_vertex_proxy_route doesn't pass _forward_headers and the default is False, so it's safe — but that's an invisible dependency holding up a security fix. Either assert it at the call site or note it where _forward_headers_for_credentialless_vertex_passthrough is defined, so someone enabling header forwarding on this route trips over it.
code-quality is the recursive_detector red on llm_request_utils.py — base drift, and #38149 fixes it.
This was referenced Aug 26, 2026
mateo-berri
added a commit
to FelipeRodriguesGare/litellm
that referenced
this pull request
Aug 26, 2026
…ertex passthrough PR BerriAI#38114 dropped whichever header user_api_key_auth would read the caller's key from, by precedence. Under custom_auth, JWT auth, or no master key that header is the caller's own Google token, so the bring-your-own-credentials Vertex branch answered 401 to every valid request. A header value is now dropped only when it is the master key or when its hash is the api_key that authenticated the request, so a Google token that auth never consumed keeps flowing while a LiteLLM key still never reaches Google. test_passthrough_post_call_guardrails.py no longer plants a MagicMock proxy_server module in sys.modules at import, which poisoned sibling tests that read module globals at call time.
doonga
pushed a commit
to greyrock-labs/home-ops
that referenced
this pull request
Sep 6, 2026
![renovate[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
…00.0) (#124) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.99.1` → `v1.100.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.100.0`](https://github.com/BerriAI/litellm/releases/tag/v1.100.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.99.1...v1.100.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.100.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.100.0/cosign.pub \ ghcr.io/berriai/litellm:v1.100.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - fix(responses): keep the conversation when chaining previous\_response\_id on the bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​37956](https://github.com/BerriAI/litellm/pull/37956) - feat(newrelic): per-team New Relic trace routing via team callbacks by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​37603](https://github.com/BerriAI/litellm/pull/37603) - perf(ci): cache uv dependencies in the lint job by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37783](https://github.com/BerriAI/litellm/pull/37783) - perf(ci): fan the budget checkers out across cores by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37784](https://github.com/BerriAI/litellm/pull/37784) - ci: port the Postgres suites off CircleCI onto service containers by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37785](https://github.com/BerriAI/litellm/pull/37785) - feat(ci): gate patching of SDK internals in tests as TQ008 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37787](https://github.com/BerriAI/litellm/pull/37787) - ci: measure enterprise/ coverage by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37788](https://github.com/BerriAI/litellm/pull/37788) - ci: run the keyless caching tests that ran in no job by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37790](https://github.com/BerriAI/litellm/pull/37790) - fix(ci): run the migration DDL guard, and stop it reading comments as SQL by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37791](https://github.com/BerriAI/litellm/pull/37791) - ci: run the enterprise package suite in GitHub Actions by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37798](https://github.com/BerriAI/litellm/pull/37798) - perf(ci): give the two longest unit shards the runner's spare cores by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37804](https://github.com/BerriAI/litellm/pull/37804) - test(exception-mapping): pin the status and error-shape table every provider maps to by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37807](https://github.com/BerriAI/litellm/pull/37807) - fix(terraform): add soft\_budget, tags, and soft\_budget\_alerting\_emails to litellm\_team by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37918](https://github.com/BerriAI/litellm/pull/37918) - fix(ui): theme the created-key box so it follows dark mode by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37985](https://github.com/BerriAI/litellm/pull/37985) - fix(ui): restore the public model name tooltip layout in the add model flow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37986](https://github.com/BerriAI/litellm/pull/37986) - fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37916](https://github.com/BerriAI/litellm/pull/37916) - fix(ui): repoint the key detail URL to the rotated hash after regenerating by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37968](https://github.com/BerriAI/litellm/pull/37968) - fix(ui): make playground chat bubbles theme-aware by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​37978](https://github.com/BerriAI/litellm/pull/37978) - fix(UI): correct skill install command and marketplace setup UX by [@​ozolam](https://github.com/ozolam) in [#​33514](https://github.com/BerriAI/litellm/pull/33514) - fix(proxy): skip health checks for strategy routers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37966](https://github.com/BerriAI/litellm/pull/37966) - fix(databricks): bill cached tokens at cache rates and add missing Claude pricing by [@​mateo-berri](https://github.com/mateo-berri) in [#​37975](https://github.com/BerriAI/litellm/pull/37975) - fix(anthropic): round-trip thinking blocks to OpenAI backends on /v1/messages by [@​mateo-berri](https://github.com/mateo-berri) in [#​37953](https://github.com/BerriAI/litellm/pull/37953) - fix(a2a): normalize agent card protocolBinding casing before transport match by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37917](https://github.com/BerriAI/litellm/pull/37917) - fix(interactions): track cost and spend for Google Interactions API requests by [@​mateo-berri](https://github.com/mateo-berri) in [#​33310](https://github.com/BerriAI/litellm/pull/33310) - fix(bedrock): stop emitting an empty assistant delta after the finish\_reason chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​36806](https://github.com/BerriAI/litellm/pull/36806) - fix(anthropic): reconcile enum with declared type in output\_format schema by [@​dkindlund](https://github.com/dkindlund) in [#​37882](https://github.com/BerriAI/litellm/pull/37882) - feat(azure\_ai): support entra id / oauth auth on every azure ai foundry route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​35415](https://github.com/BerriAI/litellm/pull/35415) - fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37982](https://github.com/BerriAI/litellm/pull/37982) - fix(proxy): parse form-encoded video edit/extension bodies after auth by [@​Souravrajvi0](https://github.com/Souravrajvi0) in [#​36513](https://github.com/BerriAI/litellm/pull/36513) - fix(anthropic): keep legacy thinking budget\_tokens on Claude 4.6 models on /v1/messages by [@​mateo-berri](https://github.com/mateo-berri) in [#​38108](https://github.com/BerriAI/litellm/pull/38108) - fix(utils): make prompt\_token\_calculator count claude models again by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38130](https://github.com/BerriAI/litellm/pull/38130) - fix(proxy): keep every value of a repeated form key, and gate the tests that hid it by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​37908](https://github.com/BerriAI/litellm/pull/37908) - fix(health): apply model\_info.health\_check\_params to health check probes by [@​mateo-berri](https://github.com/mateo-berri) in [#​38101](https://github.com/BerriAI/litellm/pull/38101) - fix(runwayml): route every generation endpoint and fix video cost tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​38115](https://github.com/BerriAI/litellm/pull/38115) - fix(passthrough): attribute spend and release budget reservation on router-model /vllm and /azure routes by [@​mateo-berri](https://github.com/mateo-berri) in [#​38111](https://github.com/BerriAI/litellm/pull/38111) - fix: match OpenAI SDK wire format on image/video routes by [@​mateo-berri](https://github.com/mateo-berri) in [#​38104](https://github.com/BerriAI/litellm/pull/38104) - fix(ci): give three unit shards a job deadline that outlasts their pytest budget by [@​tin-berri](https://github.com/tin-berri) in [#​38139](https://github.com/BerriAI/litellm/pull/38139) - feat(ui): add Gemini Family auto-router preset by [@​tin-berri](https://github.com/tin-berri) in [#​38138](https://github.com/BerriAI/litellm/pull/38138) - fix(logging\_worker): carry queued tasks across event-loop change instead of dropping them by [@​mateo-berri](https://github.com/mateo-berri) in [#​38144](https://github.com/BerriAI/litellm/pull/38144) - feat(proxy): enforce vector-store upload security controls on /v1/rag/ingest by [@​mateo-berri](https://github.com/mateo-berri) in [#​38135](https://github.com/BerriAI/litellm/pull/38135) - test(e2e): pin require\_managed\_files enforcement behind a marker-gated stack phase by [@​mateo-berri](https://github.com/mateo-berri) in [#​38117](https://github.com/BerriAI/litellm/pull/38117) - refactor(ui): move the dashboard onto class-variance-authority by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38125](https://github.com/BerriAI/litellm/pull/38125) - refactor(utils)!: delete prompt\_token\_calculator by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38132](https://github.com/BerriAI/litellm/pull/38132) - fix(auto-router): list configured auto-routers in the usage picker before they have traffic by [@​tin-berri](https://github.com/tin-berri) in [#​38129](https://github.com/BerriAI/litellm/pull/38129) - refactor(ui): install the shadcn field primitive by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38126](https://github.com/BerriAI/litellm/pull/38126) - fix(complexity\_router): keep both ends of a clipped classifier context turn by [@​tin-berri](https://github.com/tin-berri) in [#​38141](https://github.com/BerriAI/litellm/pull/38141) - fix(ci): ignore-list recursive form-field flatteners in recursive\_detector by [@​mateo-berri](https://github.com/mateo-berri) in [#​38149](https://github.com/BerriAI/litellm/pull/38149) - fix(passthrough): stop leaking the caller's virtual key on credential-less Vertex passthrough by [@​mateo-berri](https://github.com/mateo-berri) in [#​38114](https://github.com/BerriAI/litellm/pull/38114) - fix(router): stop copying forwarded credentials into retry breadcrumbs by [@​mateo-berri](https://github.com/mateo-berri) in [#​38133](https://github.com/BerriAI/litellm/pull/38133) - feat(e2e): record and replay streamed provider responses chunk-for-chunk by [@​mateo-berri](https://github.com/mateo-berri) in [#​38136](https://github.com/BerriAI/litellm/pull/38136) - fix: tolerate stream chunks without a choices key in stream\_chunk\_builder by [@​AkshaySasi](https://github.com/AkshaySasi) in [#​34382](https://github.com/BerriAI/litellm/pull/34382) - fix(files): decode x-litellm-model encoded file\_id in chat + responses by [@​hclsys](https://github.com/hclsys) in [#​29832](https://github.com/BerriAI/litellm/pull/29832) - fix(videos): forward uploaded source file on /v1/videos/edits to the provider by [@​mateo-berri](https://github.com/mateo-berri) in [#​38155](https://github.com/BerriAI/litellm/pull/38155) - fix(s3\_v2): percent-encode object keys once so signed and sent URLs match by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38005](https://github.com/BerriAI/litellm/pull/38005) - feat(ui): add error-code drilldown for failed requests on caching page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38156](https://github.com/BerriAI/litellm/pull/38156) - feat(search): add Grounding with Bing Search (bing\_grounding) as a search provider by [@​mateo-berri](https://github.com/mateo-berri) in [#​38119](https://github.com/BerriAI/litellm/pull/38119) - ci: ban row-rewriting DML from prisma migrations by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​37899](https://github.com/BerriAI/litellm/pull/37899) - fix(langsmith): keep root-run ids self-consistent so batch ingest stops rejecting header-tagged requests by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38116](https://github.com/BerriAI/litellm/pull/38116) - ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress by [@​mateo-berri](https://github.com/mateo-berri) in [#​38163](https://github.com/BerriAI/litellm/pull/38163) - chore(codeowners): unown ui container plumbing and generated files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38124](https://github.com/BerriAI/litellm/pull/38124) - fix(logging): skip parsing redacted tool call arguments by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38169](https://github.com/BerriAI/litellm/pull/38169) - feat(complexity\_router): bound the classifier context block, not each turn in it by [@​tin-berri](https://github.com/tin-berri) in [#​38145](https://github.com/BerriAI/litellm/pull/38145) - fix(http\_handler): dispose aiohttp session when AsyncHTTPHandler is finalized without a running loop by [@​anmolg1997](https://github.com/anmolg1997) in [#​36670](https://github.com/BerriAI/litellm/pull/36670) - fix(proxy): reset a stuck team member's budget by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​37971](https://github.com/BerriAI/litellm/pull/37971) - fix(anthropic/bedrock): request summarized adaptive thinking for reasoning\_effort and use provider thinking token counts by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37979](https://github.com/BerriAI/litellm/pull/37979) - fix(completion\_extras): forward reasoning\_effort=max through the Responses API bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​38222](https://github.com/BerriAI/litellm/pull/38222) - feat(vertex\_ai): add native Vertex AI Interactions API support by [@​mateo-berri](https://github.com/mateo-berri) in [#​38229](https://github.com/BerriAI/litellm/pull/38229) - test(mcp): drain the logging worker after each test so queued callbacks cannot leak into the next test by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38228](https://github.com/BerriAI/litellm/pull/38228) - fix(ui): forward OAuth issuer/authorization/token/registration URLs from the MCP server edit form by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​38154](https://github.com/BerriAI/litellm/pull/38154) - fix(together\_ai): default endpoints to api.together.ai instead of api.together.xyz by [@​mateo-berri](https://github.com/mateo-berri) in [#​38233](https://github.com/BerriAI/litellm/pull/38233) - fix(bedrock\_mantle): register a Bedrock runtime passthrough config so /bedrock/model/<deployment>/invoke works by [@​mateo-berri](https://github.com/mateo-berri) in [#​38231](https://github.com/BerriAI/litellm/pull/38231) - fix(router): resolve provider from api\_base in deployment validation and acompletion by [@​mateo-berri](https://github.com/mateo-berri) in [#​38235](https://github.com/BerriAI/litellm/pull/38235) - fix(model\_prices): raise bedrock\_mantle gpt-5.6 max\_input\_tokens to Mantle's enforced [`1050000`](https://github.com/BerriAI/litellm/commit/1050000) by [@​mateo-berri](https://github.com/mateo-berri) in [#​38225](https://github.com/BerriAI/litellm/pull/38225) - fix(bedrock\_mantle): normalize Codex input item types Mantle rejects by [@​mateo-berri](https://github.com/mateo-berri) in [#​38227](https://github.com/BerriAI/litellm/pull/38227) - fix(proxy): store the actual selected model in spend logs for Azure Model Router by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37770](https://github.com/BerriAI/litellm/pull/37770) - feat(router): per-group supported reasoning efforts with the max level by [@​tin-berri](https://github.com/tin-berri) in [#​37897](https://github.com/BerriAI/litellm/pull/37897) - fix(proxy): stop expected 4xx responses from saturating worker CPU on failure logging by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38102](https://github.com/BerriAI/litellm/pull/38102) - fix(caching): use upstream RedisCluster on redis-py with per-connection recovery by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38171](https://github.com/BerriAI/litellm/pull/38171) - perf(auth): drop guaranteed-miss internal-cache Redis read from team object lookup by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38073](https://github.com/BerriAI/litellm/pull/38073) - fix(together\_ai): route chat completions through a dedicated TogetherAIChatConfig by [@​mateo-berri](https://github.com/mateo-berri) in [#​38248](https://github.com/BerriAI/litellm/pull/38248) - fix(ui): read reasoning tokens from Responses API output\_tokens\_details by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​37952](https://github.com/BerriAI/litellm/pull/37952) - fix(dashboard): don't show a stale provider prompt-cache chip on a response-cache hit by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​37951](https://github.com/BerriAI/litellm/pull/37951) - fix(ui): render tag-based guardrail mode instead of crashing the guardrails page by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37493](https://github.com/BerriAI/litellm/pull/37493) - fix(scim): return user\_id as Group members\[].value on transformed group responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38161](https://github.com/BerriAI/litellm/pull/38161) - fix(scim): preserve existing team memberships when POST /Users adoption carries no groups by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38166](https://github.com/BerriAI/litellm/pull/38166) - fix(router): support mid-stream fallback for anthropic\_messages route type by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​38153](https://github.com/BerriAI/litellm/pull/38153) - fix(auth): support wildcard prefixes in jwt team\_allowed\_routes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37756](https://github.com/BerriAI/litellm/pull/37756) - feat(models): add missing Together AI serverless models to the cost map by [@​mateo-berri](https://github.com/mateo-berri) in [#​38230](https://github.com/BerriAI/litellm/pull/38230) - fix(cerebras)!: add max\_retries and extra\_headers to get\_supported\_openai\_params by [@​deepanshululla](https://github.com/deepanshululla) in [#​36601](https://github.com/BerriAI/litellm/pull/36601) - fix(anthropic): translate tool\_result document blocks in the /v1/messages bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​38251](https://github.com/BerriAI/litellm/pull/38251) - fix(team): serialize member\_add, member\_delete, and delete under the team's advisory lock by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​37969](https://github.com/BerriAI/litellm/pull/37969) - docs(pr-template): split Caveats bullets into severity tiers and call for plain engineering language by [@​mateo-berri](https://github.com/mateo-berri) in [#​38252](https://github.com/BerriAI/litellm/pull/38252) - fix(anthropic): carry tool\_result document blocks through the /v1/messages responses bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​38261](https://github.com/BerriAI/litellm/pull/38261) - fix(together\_ai): pass tools through for models missing from the registry by [@​mateo-berri](https://github.com/mateo-berri) in [#​38265](https://github.com/BerriAI/litellm/pull/38265) - fix(anthropic): carry user-content document blocks through the /v1/messages responses bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​38267](https://github.com/BerriAI/litellm/pull/38267) - fix(rerank): emit latency and cost headers on /rerank by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​35419](https://github.com/BerriAI/litellm/pull/35419) - perf(streaming): add shared JSONFragmentAccumulator for Vertex and Anthropic by [@​deepanshululla](https://github.com/deepanshululla) in [#​36610](https://github.com/BerriAI/litellm/pull/36610) - fix(together\_ai): strip internal thinking fields from outbound messages, keep reasoning\_content by [@​mateo-berri](https://github.com/mateo-berri) in [#​38275](https://github.com/BerriAI/litellm/pull/38275) - fix(router): persist attempted\_fallbacks and original\_model\_group into spend logs metadata by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38107](https://github.com/BerriAI/litellm/pull/38107) - fix(logging): redact tool call arguments to valid JSON and preserve null content by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38182](https://github.com/BerriAI/litellm/pull/38182) - fix(ui): stack policy flow builder below the popup layer so guardrail options render by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38273](https://github.com/BerriAI/litellm/pull/38273) - test: gate the test tree on B003 so a test cannot swap os.environ for a plain dict by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38274](https://github.com/BerriAI/litellm/pull/38274) - refactor(repositories): type prisma table access with one generic protocol by [@​mateo-berri](https://github.com/mateo-berri) in [#​38205](https://github.com/BerriAI/litellm/pull/38205) - fix(anthropic): buffer streamed responses carrying server-fulfilled tools so retrieval tool calls never reach the client by [@​mateo-berri](https://github.com/mateo-berri) in [#​36245](https://github.com/BerriAI/litellm/pull/36245) - test(together\_ai): regression suite across chat, responses, and messages surfaces by [@​mateo-berri](https://github.com/mateo-berri) in [#​38283](https://github.com/BerriAI/litellm/pull/38283) - feat(logging): add async\_post\_call\_failure\_deployment\_hook by [@​deepanshululla](https://github.com/deepanshululla) in [#​36657](https://github.com/BerriAI/litellm/pull/36657) - chore: bump litellm-enterprise 0.1.59 -> 0.1.60, litellm 1.99.0 -> 1.100.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38243](https://github.com/BerriAI/litellm/pull/38243) - test(e2e): cover Together AI reasoning, tool calls, template kwargs, and cost through a live proxy by [@​mateo-berri](https://github.com/mateo-berri) in [#​38286](https://github.com/BerriAI/litellm/pull/38286) - fix(logging): keep tracebacks for provider-originated 4xx errors by [@​mateo-berri](https://github.com/mateo-berri) in [#​38296](https://github.com/BerriAI/litellm/pull/38296) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38293](https://github.com/BerriAI/litellm/pull/38293) - refactor(ui): install the shadcn alert primitive by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38300](https://github.com/BerriAI/litellm/pull/38300) - refactor(ui): re-pull label, textarea, separator and skeleton from the registry by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38302](https://github.com/BerriAI/litellm/pull/38302) - feat(prometheus): configure deployment caller identity by [@​mphilippnv](https://github.com/mphilippnv) in [#​38221](https://github.com/BerriAI/litellm/pull/38221) - test(e2e): let the Together replayed-reasoning case survive a single provider miss by [@​mateo-berri](https://github.com/mateo-berri) in [#​38314](https://github.com/BerriAI/litellm/pull/38314) - fix(otel): map /v1/messages provider errors before failure logging by [@​mateo-berri](https://github.com/mateo-berri) in [#​38310](https://github.com/BerriAI/litellm/pull/38310) - fix(exceptions): map upstream status codes for providers with no exception\_type branch by [@​mateo-berri](https://github.com/mateo-berri) in [#​38318](https://github.com/BerriAI/litellm/pull/38318) - fix(passthrough): record ownership of streamed responses under managed ids by [@​mateo-berri](https://github.com/mateo-berri) in [#​38320](https://github.com/BerriAI/litellm/pull/38320) - fix(proxy): encrypt streamed responses ids on /openai/v1/responses and /responses aliases by [@​mateo-berri](https://github.com/mateo-berri) in [#​38325](https://github.com/BerriAI/litellm/pull/38325) - test(cost-calc): pin the rate fallbacks inside a tiered-pricing tier by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38327](https://github.com/BerriAI/litellm/pull/38327) - test(e2e): cover the Bedrock provider-feature cells customers run by [@​mateo-berri](https://github.com/mateo-berri) in [#​38232](https://github.com/BerriAI/litellm/pull/38232) - fix(together\_ai): fail open on response\_format instead of dropping it for unregistered models by [@​mateo-berri](https://github.com/mateo-berri) in [#​38269](https://github.com/BerriAI/litellm/pull/38269) - fix(proxy): honor DATABASE\_DISABLE\_PREPARED\_STATEMENTS in componentized entrypoints by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38363](https://github.com/BerriAI/litellm/pull/38363) - fix(anthropic-responses): preserve structured output strictness by [@​eugene-yao-zocdoc](https://github.com/eugene-yao-zocdoc) in [#​38211](https://github.com/BerriAI/litellm/pull/38211) - chore(typing): roll up the daily tech debt cleanups from Aug 20 to Aug 26 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37922](https://github.com/BerriAI/litellm/pull/37922) - fix(router): resolve hidden aliases for explicit lookup by [@​daniel-meismer-zocdoc](https://github.com/daniel-meismer-zocdoc) in [#​38272](https://github.com/BerriAI/litellm/pull/38272) - fix(ui): keep focus in the add model public name input while typing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38366](https://github.com/BerriAI/litellm/pull/38366) - fix(model\_prices): price 1-hour cache writes on claude-3-haiku and claude-3-opus at 2x input by [@​mateo-berri](https://github.com/mateo-berri) in [#​38371](https://github.com/BerriAI/litellm/pull/38371) - fix(proxy): keep the caller's Google token on credential-less Vertex passthrough under custom auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​38299](https://github.com/BerriAI/litellm/pull/38299) - fix(mcp): preserve provider access token lifetime by [@​daniel-meismer-zocdoc](https://github.com/daniel-meismer-zocdoc) in [#​38271](https://github.com/BerriAI/litellm/pull/38271) - chore(ui): remove stale "New" badges from the dashboard by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38374](https://github.com/BerriAI/litellm/pull/38374) - test(cost-estimate): pin the prices and period totals /cost/estimate returns by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38315](https://github.com/BerriAI/litellm/pull/38315) - fix(ci): let the mutation workflow find covered lines so it generates mutants by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38305](https://github.com/BerriAI/litellm/pull/38305) - fix(model\_prices): raise bedrock\_mantle gpt-5.5 and gpt-5.4 max\_input\_tokens to Mantle's enforced [`1050000`](https://github.com/BerriAI/litellm/commit/1050000) by [@​mateo-berri](https://github.com/mateo-berri) in [#​38368](https://github.com/BerriAI/litellm/pull/38368) - fix(azure/realtime): authenticate realtime websocket with Azure AD token when no api-key by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​34658](https://github.com/BerriAI/litellm/pull/34658) - fix(bedrock): map reasoning\_effort to reasoning.effort for OpenAI GPT-5.x on Converse by [@​6matt](https://github.com/6matt) in [#​38279](https://github.com/BerriAI/litellm/pull/38279) - test(prometheus): cover caller-identity config failure cases by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38380](https://github.com/BerriAI/litellm/pull/38380) - fix(redis): support credential providers across clients by [@​eugene-yao-zocdoc](https://github.com/eugene-yao-zocdoc) in [#​38094](https://github.com/BerriAI/litellm/pull/38094) - fix(health): support `mode: image_edit` in health checks by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38291](https://github.com/BerriAI/litellm/pull/38291) - fix(cost-map): add US data residency uplift to claude-sonnet-4-6 and mythos entries by [@​mateo-berri](https://github.com/mateo-berri) in [#​38369](https://github.com/BerriAI/litellm/pull/38369) - fix(anthropic): raise missing-credential error on /v1/messages passthrough by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38240](https://github.com/BerriAI/litellm/pull/38240) - fix(mcp): complete DCR bridge OAuth challenges by [@​daniel-meismer-zocdoc](https://github.com/daniel-meismer-zocdoc) in [#​37384](https://github.com/BerriAI/litellm/pull/37384) - test(proxy): pin the request-validation contracts in proxy/\_types.py by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38307](https://github.com/BerriAI/litellm/pull/38307) - docs(CLAUDE.md): add pull-before-work rule by [@​mateo-berri](https://github.com/mateo-berri) in [#​38386](https://github.com/BerriAI/litellm/pull/38386) - fix(anthropic): scale cache costs by fast mode and trust served speed by [@​mateo-berri](https://github.com/mateo-berri) in [#​38378](https://github.com/BerriAI/litellm/pull/38378) - fix(pricing): add azure gpt-5.6 cache write rates and correct data zone priority by [@​mateo-berri](https://github.com/mateo-berri) in [#​38370](https://github.com/BerriAI/litellm/pull/38370) - docs: tighten the pull-before-work rule in CLAUDE.md by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38389](https://github.com/BerriAI/litellm/pull/38389) - fix(health): strip credential fields from GET /health output by [@​Siraj637909](https://github.com/Siraj637909) in [#​37090](https://github.com/BerriAI/litellm/pull/37090) - fix(minimax): attach MINIMAX\_API\_KEY on anthropic messages requests by [@​mateo-berri](https://github.com/mateo-berri) in [#​38393](https://github.com/BerriAI/litellm/pull/38393) - refactor(ui): replace hand-picked z-index values with one named scale and lint it by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38282](https://github.com/BerriAI/litellm/pull/38282) - fix(health): probe Azure GA realtime path for transcription-only models by [@​mateo-berri](https://github.com/mateo-berri) in [#​38390](https://github.com/BerriAI/litellm/pull/38390) - fix(bedrock): parse cacheDetails for Converse 1h/5m cache write cost split by [@​danielva-monday](https://github.com/danielva-monday) in [#​36762](https://github.com/BerriAI/litellm/pull/36762) - fix(caching): flush async cache writes cancelled at event loop shutdown by [@​mateo-berri](https://github.com/mateo-berri) in [#​38385](https://github.com/BerriAI/litellm/pull/38385) - fix(router): resolve model\_group\_alias before pre-routing strategy dispatch by [@​tin-berri](https://github.com/tin-berri) in [#​38382](https://github.com/BerriAI/litellm/pull/38382) - feat(proxy): enforce rpm/tpm on model add + fix validation error title in UI by [@​kunal2002](https://github.com/kunal2002) in [#​36518](https://github.com/BerriAI/litellm/pull/36518) - refactor(ui): move every page header onto the shared PageHeader by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38306](https://github.com/BerriAI/litellm/pull/38306) - fix(proxy): stop cache eviction errors from failing /key/update by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38308](https://github.com/BerriAI/litellm/pull/38308) - fix(aiohttp): honor global ssl\_verify on the aiohttp\_openai handler path by [@​mateo-berri](https://github.com/mateo-berri) in [#​38400](https://github.com/BerriAI/litellm/pull/38400) - fix(logging\_worker): rescue dequeued logging tasks lost at event loop close by [@​mateo-berri](https://github.com/mateo-berri) in [#​38394](https://github.com/BerriAI/litellm/pull/38394) - fix(caching): require the namespace delimiter when checking already-namespaced redis keys by [@​mateo-berri](https://github.com/mateo-berri) in [#​38403](https://github.com/BerriAI/litellm/pull/38403) - fix(prompts): reject keyed prompt\_data with prompt\_id and populate prompt version by [@​mateo-berri](https://github.com/mateo-berri) in [#​38404](https://github.com/BerriAI/litellm/pull/38404) - fix(cost\_calculator): resolve real cost key when model\_name alias contains '/' by [@​ksk2023](https://github.com/ksk2023) in [#​38344](https://github.com/BerriAI/litellm/pull/38344) - fix(cost-map): correct prompt\_cache\_min\_tokens for Claude Fable 5 and backfill Anthropic re-export entries by [@​mateo-berri](https://github.com/mateo-berri) in [#​38405](https://github.com/BerriAI/litellm/pull/38405) - test(azure-ai): pin the 422 retry that drops the field the provider rejected by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38309](https://github.com/BerriAI/litellm/pull/38309) - refactor(ui): read the auto-router tier set through one row list by [@​tin-berri](https://github.com/tin-berri) in [#​38408](https://github.com/BerriAI/litellm/pull/38408) - fix(proxy): stop empty DB router\_settings lists from clobbering yaml fallbacks by [@​mateo-berri](https://github.com/mateo-berri) in [#​38406](https://github.com/BerriAI/litellm/pull/38406) - fix(team): allow no-reset default budgets for team members by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37708](https://github.com/BerriAI/litellm/pull/37708) - fix: forward image content lists to DeepSeek vision models by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38397](https://github.com/BerriAI/litellm/pull/38397) - fix(fireworks\_ai): stop using the trace id as the session affinity key by [@​Hamjaster](https://github.com/Hamjaster) in [#​35754](https://github.com/BerriAI/litellm/pull/35754) - fix(gemini-realtime): keep the client's voice on Vertex AI native-audio Live by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38395](https://github.com/BerriAI/litellm/pull/38395) - fix(vertex\_ai): bill Gemini grounding per unique web search query by [@​ousamabenyounes](https://github.com/ousamabenyounes) in [#​36397](https://github.com/BerriAI/litellm/pull/36397) - fix(health): make the image\_edit health probe moderation-safe by [@​mateo-berri](https://github.com/mateo-berri) in [#​38417](https://github.com/BerriAI/litellm/pull/38417) - test: gate the test tree on fifteen assertion and handler rules it already satisfies by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38361](https://github.com/BerriAI/litellm/pull/38361) - fix(proxy): key lazy openapi stubs off registered features, not sys.modules by [@​mateo-berri](https://github.com/mateo-berri) in [#​38416](https://github.com/BerriAI/litellm/pull/38416) - fix(proxy): derive auto-router health from its underlying models by [@​tin-berri](https://github.com/tin-berri) in [#​38174](https://github.com/BerriAI/litellm/pull/38174) - fix(responses): let cache-control injection reach the system prompt from instructions by [@​tin-berri](https://github.com/tin-berri) in [#​38120](https://github.com/BerriAI/litellm/pull/38120) - fix(gemini): bill Google Maps grounding as its own SKU by [@​mateo-berri](https://github.com/mateo-berri) in [#​38418](https://github.com/BerriAI/litellm/pull/38418) - fix(speech): keep proxy metadata and completion cost through the TTS completion bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​38414](https://github.com/BerriAI/litellm/pull/38414) - fix: map Gemini ON\_DEMAND\_FLEX traffic type to flex service tier by [@​bisma-nawaz](https://github.com/bisma-nawaz) in [#​37724](https://github.com/BerriAI/litellm/pull/37724) - feat(langfuse): support langfuse\_environment as a per-key dynamic callback param by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38264](https://github.com/BerriAI/litellm/pull/38264) - feat(proxy): hide unhealthy models from model listings, opt-in by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38313](https://github.com/BerriAI/litellm/pull/38313) - fix(mcp): honor admin-entered OAuth URLs on authorize after issuer yield by [@​mateo-berri](https://github.com/mateo-berri) in [#​38379](https://github.com/BerriAI/litellm/pull/38379) - fix(model\_prices): correct gemini-3.5-flash-lite flex cache-read pricing by [@​mateo-berri](https://github.com/mateo-berri) in [#​38422](https://github.com/BerriAI/litellm/pull/38422) - fix(cost): price gemini-live-2.5-flash-native-audio realtime sessions by [@​mateo-berri](https://github.com/mateo-berri) in [#​38419](https://github.com/BerriAI/litellm/pull/38419) - fix(cost-map): correct Gemini TTS and native-audio rates by [@​mateo-berri](https://github.com/mateo-berri) in [#​38412](https://github.com/BerriAI/litellm/pull/38412) - fix(prompts): propagate PATCHed prompt templates to every worker and pod by [@​mateo-berri](https://github.com/mateo-berri) in [#​38411](https://github.com/BerriAI/litellm/pull/38411) - fix(model\_prices): bill gemini -latest/preview alias cache reads at 10% of input by [@​mateo-berri](https://github.com/mateo-berri) in [#​38423](https://github.com/BerriAI/litellm/pull/38423) - fix(proxy): sync search tools into the router on management writes by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38392](https://github.com/BerriAI/litellm/pull/38392) - feat(guardrails): track Azure Prompt Shield usage and cost with spend isolation by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38387](https://github.com/BerriAI/litellm/pull/38387) - fix(prompts): apply prompt templates before routing on /v1/responses and honor ignore\_prompt\_manager\_model by [@​mateo-berri](https://github.com/mateo-berri) in [#​38407](https://github.com/BerriAI/litellm/pull/38407) - fix(cost): make cost-breakdown headers respect service tier by [@​mateo-berri](https://github.com/mateo-berri) in [#​38424](https://github.com/BerriAI/litellm/pull/38424) - fix(mcp): add litellm\[mcp] extra and actionable error when streamable\_http\_client is missing by [@​mateo-berri](https://github.com/mateo-berri) in [#​38399](https://github.com/BerriAI/litellm/pull/38399) - revert(proxy): remove router\_model\_name from auto-routed response bodies by [@​tin-berri](https://github.com/tin-berri) in [#​38429](https://github.com/BerriAI/litellm/pull/38429) - fix(google\_genai): price streamed generateContent with the provider that served it by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​36055](https://github.com/BerriAI/litellm/pull/36055) - fix(logging): stop billing and logging response reads as LLM calls by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​36890](https://github.com/BerriAI/litellm/pull/36890) - fix(budget): serialize model\_max\_budget before the /budget/update write by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38430](https://github.com/BerriAI/litellm/pull/38430) - fix(ui): block the auto-router submit on a missing classifier model and an orphaned keyword rule by [@​tin-berri](https://github.com/tin-berri) in [#​38427](https://github.com/BerriAI/litellm/pull/38427) - feat(complexity\_router): heuristic-first classifier chaining by [@​tin-berri](https://github.com/tin-berri) in [#​38428](https://github.com/BerriAI/litellm/pull/38428) - test(e2e): un-skip the per-model budget update case by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38437](https://github.com/BerriAI/litellm/pull/38437) - fix(cost): stop double-billing cached tokens that overlap a modality by [@​Srivatsa03](https://github.com/Srivatsa03) in [#​37407](https://github.com/BerriAI/litellm/pull/37407) - feat(ui): add Teams list CSV export with budgets, model grants, and rate limits by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38436](https://github.com/BerriAI/litellm/pull/38436) - fix(mcp): accept raw x-litellm-api-key on streamable HTTP admission by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38364](https://github.com/BerriAI/litellm/pull/38364) - fix: bound row count on GET /spend/logs to stop unbounded LiteLLM\_SpendLogs scans by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38420](https://github.com/BerriAI/litellm/pull/38420) - fix(scim): apply default\_team\_params (incl. models) to SCIM-created teams by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38433](https://github.com/BerriAI/litellm/pull/38433) - fix(prompts): propagate prompt deletes to every worker and pod by [@​mateo-berri](https://github.com/mateo-berri) in [#​38434](https://github.com/BerriAI/litellm/pull/38434) - fix(anthropic\_adapter): carry web search cost into /v1/messages breakdown headers by [@​mateo-berri](https://github.com/mateo-berri) in [#​38439](https://github.com/BerriAI/litellm/pull/38439) - fix(ui): show custom technical keywords on every router whose scorer runs by [@​tin-berri](https://github.com/tin-berri) in [#​38451](https://github.com/BerriAI/litellm/pull/38451) - fix(e2e): move the vertex realtime suite off the retired Live preview model by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38454](https://github.com/BerriAI/litellm/pull/38454) - feat(newrelic): per-team cost and usage metrics via team callbacks by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​37610](https://github.com/BerriAI/litellm/pull/37610) - fix(ui): carry a preset's per-tier litellm\_params through the prefill by [@​tin-berri](https://github.com/tin-berri) in [#​38453](https://github.com/BerriAI/litellm/pull/38453) - fix(e2e): size the mid-conversation-system cache prefix above the minimum deterministically by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38468](https://github.com/BerriAI/litellm/pull/38468) - fix(e2e): disable thinking on the gemini chat cost test instead of racing its budget by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38469](https://github.com/BerriAI/litellm/pull/38469) - feat(ui): put the auto-router savings hero on a spend rail and a four-tile row by [@​tin-berri](https://github.com/tin-berri) in [#​38470](https://github.com/BerriAI/litellm/pull/38470) - feat(ui): toggle internal health check visibility in request logs by [@​mateo-berri](https://github.com/mateo-berri) in [#​38391](https://github.com/BerriAI/litellm/pull/38391) - fix(mcp): canonicalize bearer scheme on bridge egress by [@​daniel-meismer-zocdoc](https://github.com/daniel-meismer-zocdoc) in [#​38398](https://github.com/BerriAI/litellm/pull/38398) - refactor: clean up fresh tech debt from 2026-08-27 window by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38484](https://github.com/BerriAI/litellm/pull/38484) - fix(exception\_mapping\_utils): map unmapped exceptions when model and provider are unset by [@​mateo-berri](https://github.com/mateo-berri) in [#​38496](https://github.com/BerriAI/litellm/pull/38496) - fix(ui\_sso): resolve highest privilege Entra app role, not first in claim by [@​imranismail](https://github.com/imranismail) in [#​36728](https://github.com/BerriAI/litellm/pull/36728) - fix(proxy): regenerate lazy OpenAPI snapshot and guard it in CI by [@​mateo-berri](https://github.com/mateo-berri) in [#​38410](https://github.com/BerriAI/litellm/pull/38410) - feat(ui): add cache hit/miss filter to Request Logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38432](https://github.com/BerriAI/litellm/pull/38432) - fix(bedrock): sign rerank requests with the shared header-filtered SigV4 helper (internal copy of [#​36462](https://github.com/BerriAI/litellm/issues/36462)) by [@​mateo-berri](https://github.com/mateo-berri) in [#​38093](https://github.com/BerriAI/litellm/pull/38093) - fix(bedrock): sign rerank requests with the shared, header-filtered SigV4 helper by [@​noahnistler](https://github.com/noahnistler) in [#​36462](https://github.com/BerriAI/litellm/pull/36462) - fix(ui): order the auto-routers table newest first so a new router lands on page one by [@​tin-berri](https://github.com/tin-berri) in [#​38545](https://github.com/BerriAI/litellm/pull/38545) - feat(ui): run the Anthropic Family preset's reasoning tier on Opus 5 at high thinking by [@​tin-berri](https://github.com/tin-berri) in [#​38490](https://github.com/BerriAI/litellm/pull/38490) - build(ui): bump nginx to 1.31-alpine by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38541](https://github.com/BerriAI/litellm/pull/38541) - feat(otel): support per-team/per-key service.name for OTel v2 destinations by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38532](https://github.com/BerriAI/litellm/pull/38532) - test(e2e): de-flake the cost-header cache read and the router fallback control by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38435](https://github.com/BerriAI/litellm/pull/38435) - feat(gemini): day-0 support for gemini-3.5-transcribe and transcribe-live by [@​mateo-berri](https://github.com/mateo-berri) in [#​38540](https://github.com/BerriAI/litellm/pull/38540) - feat(health): opt-in model-group allowlist for background health checks and health-check routing by [@​mateo-berri](https://github.com/mateo-berri) in [#​38539](https://github.com/BerriAI/litellm/pull/38539) - fix(mcp): keep upstream OAuth Authorization when jwt signer hook injects one on tools/call by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38555](https://github.com/BerriAI/litellm/pull/38555) - fix: suppress misleading register\_model unresolved-cost warnings for entries without custom pricing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38542](https://github.com/BerriAI/litellm/pull/38542) - feat(proxy): opt-in budget rollover carrying overage into the next window by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38514](https://github.com/BerriAI/litellm/pull/38514) - fix(auth): skip guaranteed-miss team lookup for the litellm-dashboard sentinel by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38471](https://github.com/BerriAI/litellm/pull/38471) - fix(key\_management): allow /key/update to keep or shrink MCP server grants the key already holds by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38463](https://github.com/BerriAI/litellm/pull/38463) - fix: keep schema reconciliation from fighting a partitioned LiteLLM\_SpendLogs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38452](https://github.com/BerriAI/litellm/pull/38452) - fix(ui): open select popups below the trigger instead of over it by [@​tin-berri](https://github.com/tin-berri) in [#​38554](https://github.com/BerriAI/litellm/pull/38554) - fix(realtime): bill trailing audio when a Gemini transcribe Live session closes by [@​mateo-berri](https://github.com/mateo-berri) in [#​38563](https://github.com/BerriAI/litellm/pull/38563) - test(e2e): cover key generate and update on the Admin UI path by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38448](https://github.com/BerriAI/litellm/pull/38448) - chore: bump litellm-enterprise 0.1.60 -> 0.1.61, litellm-proxy-extras 0.4.89 -> 0.4.90 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38566](https://github.com/BerriAI/litellm/pull/38566) - fix(ui): let the paginated search select keep what the user types by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38475](https://github.com/BerriAI/litellm/pull/38475) - fix(otel): anchor MCP tool-call spans to the gateway's own trace, link the client's context by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38317](https://github.com/BerriAI/litellm/pull/38317) - fix: roll up the open deflake fixes for the MCP logging queue, PTU rollup, license gate, and pricing test isolation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37833](https://github.com/BerriAI/litellm/pull/37833) - fix(model\_prices): rolling registry audit - verified models and rates for Novita, DeepInfra, W\&B, Bedrock Sol, Gemini, Fireworks, Azure gpt-5.6, Mistral, Together by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38207](https://github.com/BerriAI/litellm/pull/38207) - test(together\_ai): assert fail-open supported params for models missing from the registry by [@​mateo-berri](https://github.com/mateo-berri) in [#​38487](https://github.com/BerriAI/litellm/pull/38487) - test(e2e): let the together tool tests accept parallel calls by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38567](https://github.com/BerriAI/litellm/pull/38567) - feat(mcp): let a resolved OAuth token target a custom upstream header by [@​tin-berri](https://github.com/tin-berri) in [#​38456](https://github.com/BerriAI/litellm/pull/38456) - feat(together\_ai): map reasoning\_effort per model class by [@​mateo-berri](https://github.com/mateo-berri) in [#​38263](https://github.com/BerriAI/litellm/pull/38263) - feat(dashscope): support qwen-image-3.0 and qwen-image-3.0-pro image generation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38449](https://github.com/BerriAI/litellm/pull/38449) - fix(cost): apply Together AI cache read pricing and per-model registry rates by [@​mateo-berri](https://github.com/mateo-berri) in [#​38280](https://github.com/BerriAI/litellm/pull/38280) - fix(guardrails): forward aws\_external\_id when the bedrock guardrail assumes a role by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38376](https://github.com/BerriAI/litellm/pull/38376) - fix(transcription): synthesize srt/vtt output for adapters without native subtitle formats by [@​mateo-berri](https://github.com/mateo-berri) in [#​38561](https://github.com/BerriAI/litellm/pull/38561) - fix(streaming): preserve provider service-tier metadata so Vertex flex streams bill at flex rates by [@​mateo-berri](https://github.com/mateo-berri) in [#​38458](https://github.com/BerriAI/litellm/pull/38458) - fix(realtime): bill Gemini Live native-audio output tokens at the audio rate by [@​mateo-berri](https://github.com/mateo-berri) in [#​38457](https://github.com/BerriAI/litellm/pull/38457) - fix(anthropic): carry tool\_reference tool results through the guardrail translation round trip by [@​mateo-berri](https://github.com/mateo-berri) in [#​38465](https://github.com/BerriAI/litellm/pull/38465) - fix(anthropic-adapter): pass provider-native and OpenAI-format tools through on /v1/messages by [@​mateo-berri](https://github.com/mateo-berri) in [#​38431](https://github.com/BerriAI/litellm/pull/38431) - test(e2e): serve the vision image from our own fixture by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38575](https://github.com/BerriAI/litellm/pull/38575) - feat(together\_ai): add zai-org/GLM-5.3-Flash to the model registry by [@​mateo-berri](https://github.com/mateo-berri) in [#​38486](https://github.com/BerriAI/litellm/pull/38486) - fix(ui): stop server-searched comboboxes from clobbering picks and queries by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38574](https://github.com/BerriAI/litellm/pull/38574) - feat(model\_prices): let a map entry declare its exact reasoning\_effort levels by [@​tin-berri](https://github.com/tin-berri) in [#​38481](https://github.com/BerriAI/litellm/pull/38481) - fix(anthropic): carry the adaptive effort tier to every bridged Claude target by [@​tin-berri](https://github.com/tin-berri) in [#​38533](https://github.com/BerriAI/litellm/pull/38533) - feat(alerting): add native Microsoft Teams alerting destination by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38367](https://github.com/BerriAI/litellm/pull/38367) - chore(proxy): resync the generated API artifacts with the current models by [@​tin-berri](https://github.com/tin-berri) in [#​38587](https://github.com/BerriAI/litellm/pull/38587) - fix(router): reject complexity-router settings written outside complexity\_router\_config by [@​tin-berri](https://github.com/tin-berri) in [#​38570](https://github.com/BerriAI/litellm/pull/38570) - feat(ui): session-level cache observability in request logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38442](https://github.com/BerriAI/litellm/pull/38442) - fix(ui): link Virtual Keys hint through the migrated /ui route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38596](https://github.com/BerriAI/litellm/pull/38596) - fix(anthropic): carry the effort tier only where the target declares reasoning\_effort by [@​tin-berri](https://github.com/tin-berri) in [#​38592](https://github.com/BerriAI/litellm/pull/38592) - fix(presidio): chunk oversized text before /analyze so large content blocks do not fail by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38483](https://github.com/BerriAI/litellm/pull/38483) - fix(logging): stop stream-based log collectors classifying INFO logs as errors by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38476](https://github.com/BerriAI/litellm/pull/38476) - feat(ui): dry-run an auto-router config against the backend before saving it by [@​tin-berri](https://github.com/tin-berri) in [#​38595](https://github.com/BerriAI/litellm/pull/38595) - fix(guardrails): add fail-open mode to CrowdStrike AIDR guardrail by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38568](https://github.com/BerriAI/litellm/pull/38568) - fix(router): copy instead of mutating caller metadata when scrubbing fallback stamp keys by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38586](https://github.com/BerriAI/litellm/pull/38586) - feat(proxy): opt-in enforce\_fallback\_model\_access authorizes router fallbacks against the calling key by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38572](https://github.com/BerriAI/litellm/pull/38572) - fix(langfuse): warn and drop invalid LANGFUSE\_TRACING\_ENVIRONMENT instead of failing requests by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38582](https://github.com/BerriAI/litellm/pull/38582) - fix(tencent): route thinking through extra\_body in chat completions by [@​FelipeRodriguesGare](https://github.com/FelipeRodriguesGare) in [#​38100](https://github.com/BerriAI/litellm/pull/38100) - test-check-commits by [@​nickhac](https://github.com/nickhac) in [#​36344](https://github.com/BerriAI/litellm/pull/36344) - feat(proxy): dry-run a real request body on /auto\_router/test\_routing by [@​tin-berri](https://github.com/tin-berri) in [#​38590](https://github.com/BerriAI/litellm/pull/38590) - fix(shadow\_eval): refuse a judge model that also serves one of the arms it grades by [@​tin-berri](https://github.com/tin-berri) in [#​38589](https://github.com/BerriAI/litellm/pull/38589) - fix(anthropic): resolve /v1/messages effort tiers through the capability owner by [@​tin-berri](https://github.com/tin-berri) in [#​38492](https://github.com/BerriAI/litellm/pull/38492) - fix(router): fall over on raised mid-stream errors in /v1/messages streams by [@​mateo-berri](https://github.com/mateo-berri) in [#​38606](https://github.com/BerriAI/litellm/pull/38606) - feat(models): add daily Together AI model registry sync script and workflow by [@​mateo-berri](https://github.com/mateo-berri) in [#​38257](https://github.com/BerriAI/litellm/pull/38257) - feat(ui): the model and wire layer for operator-defined auto-router tier sets by [@​tin-berri](https://github.com/tin-berri) in [#​38602](https://github.com/BerriAI/litellm/pull/38602) - fix(moonshot, together\_ai): send the reasoning effort Kimi K3 accepts by [@​tin-berri](https://github.com/tin-berri) in [#​38611](https://github.com/BerriAI/litellm/pull/38611) - fix(ui): one-click theme toggle and matching Docs/Blog styling in the top bar by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​38601](https://github.com/BerriAI/litellm/pull/38601) - feat(proxy): opt-in flags to require rpm/tpm on model and project create by [@​ansh-agrawal](https://github.com/ansh-agrawal) in [#​36514](https://github.com/BerriAI/litellm/pull/36514) - fix(exceptions): keep a refused connection an APIConnectionError by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38624](https://github.com/BerriAI/litellm/pull/38624) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38616](https://github.com/BerriAI/litellm/pull/38616) - fix(anthropic): drop and self-heal empty thinking blocks on /v1/messages by [@​tin-berri](https://github.com/tin-berri) in [#​38625](https://github.com/BerriAI/litellm/pull/38625) - fix(anthropic): handle per-level reasoning\_effort flags without supports\_reasoning by [@​tin-berri](https://github.com/tin-berri) in [#​38618](https://github.com/BerriAI/litellm/pull/38618) - fix(complexity\_router): route client housekeeping calls to the cheapest tier by [@​tin-berri](https://github.com/tin-berri) in [#​38598](https://github.com/BerriAI/litellm/pull/38598) - test: fix staging CI regressions from [#​38182](https://github.com/BerriAI/litellm/issues/38182), [#​38144](https://github.com/BerriAI/litellm/issues/38144), [#​38265](https://github.com/BerriAI/litellm/issues/38265), [#​37962](https://github.com/BerriAI/litellm/issues/37962), and [#​37969](https://github.com/BerriAI/litellm/issues/37969) by [@​mateo-berri](https://github.com/mateo-berri) in [#​38304](https://github.com/BerriAI/litellm/pull/38304) - feat(spend): report prompt caching savings as total and gateway-attributed by [@​tin-berri](https://github.com/tin-berri) in [#​38134](https://github.com/BerriAI/litellm/pull/38134) - fix(proxy): let llm\_api virtual keys read /model\_group/info by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38662](https://github.com/BerriAI/litellm/pull/38662) - feat(ui): edit the auto-router tier set with custom classifier-defined tiers by [@​tin-berri](https://github.com/tin-berri) in [#​38603](https://github.com/BerriAI/litellm/pull/38603) - fix(proxy): count tools, system, and Anthropic image and document blocks in the count\_tokens fallback (internal copy of [#​36671](https://github.com/BerriAI/litellm/issues/36671)) by [@​mateo-berri](https://github.com/mateo-berri) in [#​38657](https://github.com/BerriAI/litellm/pull/38657) - test: refresh the suites that drifted from langfuse and OpenAI's retired Assistants API by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38637](https://github.com/BerriAI/litellm/pull/38637) - test(e2e): unskip four tests whose blockers no longer hold by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38640](https://github.com/BerriAI/litellm/pull/38640) - feat(proxy): add paginated GET /public/v1/model\_hub by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38636](https://github.com/BerriAI/litellm/pull/38636) - refactor(ui): type search tool params from the generated schema by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38633](https://github.com/BerriAI/litellm/pull/38633) - feat(a2a): semantic search over the agent registry via GET /v1/agents?query and an agent\_search MCP tool by [@​mateo-berri](https://github.com/mateo-berri) in [#​38609](https://github.com/BerriAI/litellm/pull/38609) - fix(model\_prices): add bedrock\_mantle gpt-5.5/5.4 272K tiers, align sol with AWS invoice by [@​mateo-berri](https://github.com/mateo-berri) in [#​38615](https://github.com/BerriAI/litellm/pull/38615) - fix(ui): keep the usage filter visible when the caller's scope is empty by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38581](https://github.com/BerriAI/litellm/pull/38581) - fix(registry): add Gemini Omni 1.1 Flash, xAI grok-imagine image models, Mistral cache-read pricing, GLM 5.3 Flash + Kimi K2.7 Code entries by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38560](https://github.com/BerriAI/litellm/pull/38560) - fix(logging): preserve null end user in callbacks by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38642](https://github.com/BerriAI/litellm/pull/38642) - test: close mutation-testing gaps in container, skills and openai-like config factories by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​38677](https://github.com/BerriAI/litellm/pull/38677) - fix: enforce MCP toolsets attached to a team, org, or internal user by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​38488](https://github.com/BerriAI/litellm/pull/38488) - fix(tests): drain the global logging worker in RAG aquery billing tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38653](https://github.com/BerriAI/litellm/pull/38653) - chore(techdebt): type new signatures and drop slop comments from the last 24h by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​38644](https://github.com/Be…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
Authorization,x-litellm-api-key,x-goog-api-key,api-key,x-api-key,Ocp-Apim-Subscription-Key, the mapped-routelitellm_user_api_keyheader, and the operator-configuredlitellm_key_header_nameHow it solves it:
SpecialHeaders.litellm_credential_header_names(), so every proxy-only auth header Google never consumes (x-litellm-api-key,api-key,x-api-key,Ocp-Apim-Subscription-Key) is dropped, plus any operator-configured caller-key header (litellm_key_header_nameand eachpass_through_endpointsentry'slitellm_user_api_key), and future additions are covered automaticallyuser_api_key_authuses and value-strip exactly that value fromAuthorization/x-goog-api-key(which can instead hold a real Google credential) and from the operator-configured custom key header, normalizing the value with the auth module's own_get_bearer_tokenso every scheme it accepts (Bearer/bearer/Basic/AWS4-HMAC-SHA256) is matchedUser Flow
Before: a developer calls Vertex passthrough on a proxy with no Vertex credential configured, and their LiteLLM key is handed to Google
https://litellm-domain/vertex_ai/v1/projects/my-proj/locations/global/publishers/google/models/gemini-2.5-pro:generateContentwithAuthorization: Bearer sk-...(their LiteLLM virtual key) and a JSON bodyhttps://aiplatform.googleapis.com/v1/projects/my-proj/...:generateContentwith that sameAuthorization: Bearer sk-...still on itx-litellm-api-key,x-goog-api-key,api-key,x-api-key, or the operator's configured key header instead leaks it the same wayAfter: the same call fails fast with a clean 401, and the key is never forwarded
Authorization: Bearer sk-...and the same body401saying no Vertex credential is configured and the virtual key is not forwardedhttps://aiplatform.googleapis.com; sending the key inx-litellm-api-keyorx-goog-api-keygives the same 401, and anyapi-key/x-api-key/ configured custom key header is stripped before forwardingAuthorization, or a real Google API key inx-goog-api-key) still has it forwarded, now with the LiteLLM key and the other proxy auth headers stripped outRelevant issues
Linear ticket
Resolves LIT-5997
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Shared setup (no real virtual key is ever sent to real Google: a local sink intercepts every egress and the capture is the proof)
DEFAULT_VERTEXAI_*env vars, and nouse_in_pass_throughmodel, so the passthrough takes the credential-less branchgoogleapis.com, records the exact headers that would have gone to Google, and returns a synthetic599so nothing reaches Google. A leak shows up as the virtual key (or any caller secret) appearing in that capturecurl -sX POST http://127.0.0.1:PORT/key/generate -H "<auth header>: Bearer $MASTER_KEY" -d '{"duration":"2h"}'returnssk-…{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}general_settings.litellm_key_header_name: x-company-key; every other case runs against a default-config proxyBefore (28b433a)
Case A: virtual key in Authorization
curl -sX POST http://127.0.0.1:49346/vertex_ai/v1/projects/my-proj/locations/global/publishers/google/models/gemini-2.5-pro:generateContent -H "Authorization: Bearer sk-…" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer sk-…, the virtual key was on its way to GoogleCase B: virtual key in x-litellm-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -d '<payload>'aiplatform.googleapis.com:x-litellm-api-key: sk-…, the virtual key was on its way to GoogleCase D: virtual key in x-goog-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: sk-…" -d '<payload>'aiplatform.googleapis.com:x-goog-api-key: sk-…andx-litellm-api-key: sk-…. The virtual key was on its way to Google, this time posing as a Google API keyCase F: distinct caller secrets in api-key and x-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "api-key: azure-secret-abc123" -H "x-api-key: anthropic-secret-def456" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauth,api-key: azure-secret-abc123,x-api-key: anthropic-secret-def456, andx-litellm-api-key: sk-…. Every proxy auth header, including two unrelated caller secrets and the virtual key, went to GoogleCase G: virtual key in the operator-configured custom key header
Against a merge-base proxy configured with
general_settings.litellm_key_header_name: x-company-keycurl -sX POST http://127.0.0.1:42826/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-company-key: Bearer sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'aiplatform.googleapis.com:x-company-key: Bearer sk-…andx-goog-api-key: AIzaSyReal-Google-Api-Key-000. The caller authenticated with the custom header, and that same header carrying the virtual key was forwarded to Google alongside the real Google keyCase H: caller Azure APIM secret in Ocp-Apim-Subscription-Key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "Ocp-Apim-Subscription-Key: azure-apim-secret-xyz789" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthandOcp-Apim-Subscription-Key: azure-apim-secret-xyz789. The caller's Azure APIM subscription key, an auth header the proxy accepts but Google never consumes, was forwarded to GoogleCase J: virtual key echoed into Authorization with a Basic scheme
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Basic sk-…" -d '<payload>'aiplatform.googleapis.com:authorization: Basic sk-…. The caller authenticated withx-litellm-api-keyand echoed the same virtual key intoAuthorizationunder aBasicscheme, and that header carrying the key was forwarded to GoogleAfter (16a81c9)
The header-filtering behavior in Cases A-H is stable across the hardening commits and was captured against the default-config proxy on port 41337 and the custom-key-header proxy on port 41779. Case I was run against a fresh default-config proxy on port 40923, and Case J plus the no-regression re-check of Cases C and E against a fresh default-config proxy on port 40611 at this exact tip.
Case A: virtual key in Authorization
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "Authorization: Bearer sk-…" -d '<payload>'HTTP 401with{"detail":"No Vertex AI credential is configured on this proxy and the request carried no upstream Google credential. The LiteLLM virtual key is not forwarded to Google. ..."}. The sink recorded nothing new: no request reachedgoogleapis.comCase B: virtual key in x-litellm-api-key
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -d '<payload>'HTTP 401with the same "no Vertex AI credential is configured" detail. The sink recorded nothing new: no request reachedgoogleapis.comCase C: bring-your-own Google token, virtual key in x-litellm-api-key for proxy auth
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauth,x-litellm-api-key: <absent>, and the virtual key is nowhere in the outbound. Real bring-your-own passthrough still works, with the key strippedCase D: virtual key in x-goog-api-key
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: sk-…" -d '<payload>'HTTP 401with the same "no Vertex AI credential is configured" detail. The sink recorded nothing new: no request reachedgoogleapis.com. The key posing as a Google API key no longer satisfies the gateCase E: bring-your-own real Google API key in x-goog-api-key, virtual key in x-litellm-api-key for proxy auth
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:x-goog-api-key: AIzaSyReal-Google-Api-Key-000,x-litellm-api-key: <absent>, and the virtual key is nowhere in the outbound. A real Google API key that differs from the virtual key still forwards, with the key strippedCase F: distinct caller secrets in api-key and x-api-key, bring-your-own Google token for the real credential
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "api-key: azure-secret-abc123" -H "x-api-key: anthropic-secret-def456" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthonly.x-litellm-api-key,api-key, andx-api-keyare all<absent>, and neitherazure-secret-abc123,anthropic-secret-def456, nor the virtual key appears anywhere. The Google token still forwards, every proxy auth header is droppedCase G: virtual key in the operator-configured custom key header
Against the proxy configured with
general_settings.litellm_key_header_name: x-company-keycurl -sw '%{http_code}' -X POST http://127.0.0.1:41779/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-company-key: Bearer sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:x-goog-api-key: AIzaSyReal-Google-Api-Key-000only,x-company-key: <absent>, and the virtual key is nowhere in the outbound. The caller still authenticates with the custom header, the real Google key still forwards, and the virtual key is strippedx-company-keywith no real Google credential returnsHTTP 401and nothing reachesgoogleapis.comCase H: caller Azure APIM secret in Ocp-Apim-Subscription-Key, legitimate Google header preserved
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "Ocp-Apim-Subscription-Key: azure-apim-secret-xyz789" -H "X-Goog-User-Project: my-billing-proj" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthandX-Goog-User-Project: my-billing-proj, whileOcp-Apim-Subscription-Keyis<absent>andazure-apim-secret-xyz789appears nowhere. The caller's Azure APIM secret is dropped, and the genuine GoogleX-Goog-User-Projectheader is preserved so real Vertex requests keep workingCase I: virtual key resolved through the full auth precedence, and no regression to real credentials
The route authenticates through
Depends(user_api_key_auth), which accepts the caller key from every header inSpecialHeaders.litellm_credential_header_names(),x-goog-api-keyincluded. The filter now resolves the caller key by that same precedence and value-strips exactly the value that authenticated, sox-goog-api-keyis stripped when it carried the key and preserved when it carried a real Google key alongside a higher-precedence virtual key. All against the port 40923 proxy at this tip:curl -sw '%{http_code}' -X POST http://127.0.0.1:40923/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-goog-api-key: sk-…" -d '<payload>'returnsHTTP 401and the sink records nothing new (the proxy's own auth rejects a virtual key inx-goog-api-keyon this route today, and the filter would strip it regardless, closing the boundary that a mocked unit test exercises directly)x-litellm-api-key: sk-…+Authorization: Bearer ya29.fake-google-oauth) still returnsHTTP 599withauthorization: Bearer ya29.fake-google-oauthforwarded and the virtual key absentx-litellm-api-key: sk-…+x-goog-api-key: AIzaSyReal-Google-Api-Key-000) still returnsHTTP 599withx-goog-api-key: AIzaSyReal-Google-Api-Key-000forwarded and the virtual key absent, so resolving by precedence does not strip a genuine Google keyCase J: virtual key echoed into Authorization with a Basic scheme, and no regression to real credentials
Against a fresh default-config proxy on port 40611 at this tip:
curl -sw '%{http_code}' -X POST http://127.0.0.1:40611/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Basic sk-…" -d '<payload>'returnsHTTP 401and the sink records nothing new. Normalizing the value with the auth module's own_get_bearer_tokenstrips theBasicscheme just as authentication does, so the echoed virtual key matches the authenticated key andAuthorizationis dropped, leaving no upstream credentialx-litellm-api-key: sk-…+Authorization: Bearer ya29.fake-google-oauth) still returnsHTTP 599withauthorization: Bearer ya29.fake-google-oauthforwarded and the virtual key absentx-litellm-api-key: sk-…+x-goog-api-key: AIzaSyReal-Google-Api-Key-000) still returnsHTTP 599withx-goog-api-key: AIzaSyReal-Google-Api-Key-000forwarded and the virtual key absentCase K: virtual key in the mapped-route litellm_user_api_key header, with real credentials preserved
The
/vertex_aiprefix is a mapped pass-through route, souser_api_key_authaccepts the caller key from a header literally namedlitellm_user_api_keyand applies it last, overriding every other source. Against fresh default-config proxies on ports 40611 (before this commit) and 40337 (this tip), same request each side:litellm_user_api_key: sk-…(auth) +Authorization: Bearer ya29.fake-google-oauth+x-goog-api-key: AIzaSyReal-Google-Api-Key-000.HTTP 599, and the sink capture toaiplatform.googleapis.comshowslitellm_user_api_key: sk-…forwarded with the virtual key, while the realAuthorizationwas dropped. The virtual key reached Google and the bring-your-own token was lostHTTP 599, and the sink capture showsauthorization: Bearer ya29.fake-google-oauthandx-goog-api-key: AIzaSyReal-Google-Api-Key-000both forwarded,litellm_user_api_key<absent>, and the virtual key nowhere. The key that authenticated is stripped, and both real credentials are preservedType
🐛 Bug Fix
Caveats (if any)
Scope of this PR is the credential-less Vertex passthrough leaking the caller's LiteLLM credential through request headers. One adjacent vector is intentionally left for a separate, focused change: a caller can also send a virtual key in the
?key=URL query parameter (the Google AI Studio auth convention thatuser_api_key_authreads on generateContent routes). A client that authenticates only with?key=is already rejected with a 401 on this branch (no surviving upstream Google credential), so that common case does not leak. The virtual key does still ride the forwarded URL query when the caller both authenticates with a header and brings a real Google credential, but stripping credential query params belongs in the shared pass-through URL-forwarding path (it affects every provider, not just Vertex) and is being tracked as its own follow-up rather than widening this PR's blast radius.Final Attestation
Live PR risk
/live-pr-risk CHECKED e6eb6a4: SAFE. The graph is tiny and fully local:
_forwarded_headers_for_credentialless_vertex_passthroughis called only by_prepare_vertex_auth_headers, which is called only by_base_vertex_proxy_route, which is reached by the two public routesvertex_proxy_routeandvertex_discovery_proxy_route. The re-signed_prepare_vertex_auth_headersnow returnsMapping[str, str]and its credential-less branch raisesHTTPException(401); nothing downstream mutates the returned headers, andcreate_pass_through_routecopies them withdict(...), so the immutableMappingProxyTypeis safe on every path.vertex_proxy_routewas driven live in the proof above;vertex_discovery_proxy_routeruns the identical credential-less branch and is covered by unit tests. Direct-caller and related passthrough tests pass on the fixed head (160 passed), includingtest_vertex_passthrough_load_balancing.py, which unpacks the new tuple directly./live-pr-risk CHECKED 4bc0977: SAFE. This commit hardens the same branch to strip the virtual key from every forwarded header by value (normalizing any
Bearerprefix) instead of only fromAuthorizationby name, closing thex-goog-api-keyvector Greptile flagged. Re-walked the graph for the added code: the new_bearer_strippedis a pure module-private helper referenced only inside_forwarded_headers_for_credentialless_vertex_passthrough, and that function still has exactly one caller, so the blast radius is identical to the commit above. No new dependents, side effects, or raises./live-pr-risk CHECKED e7c2ede: SAFE. This commit additionally drops the proxy-only auth headers Google never consumes (
x-litellm-api-key,api-key,x-api-key) by name via a new module-private_HEADERS_NEVER_FORWARDED_TO_VERTEXfrozenset, keeping the by-value virtual-key strip forAuthorization/x-goog-api-key. The frozenset is referenced only inside_forwarded_headers_for_credentialless_vertex_passthrough, which still has exactly one caller, so the blast radius is unchanged; no new dependents, side effects, or raises, and the gate is untouched./live-pr-risk CHECKED ee03632: SAFE. This commit closes the last vector Greptile flagged:
user_api_key_authalso authenticates a caller from the operator-configuredgeneral_settings.litellm_key_header_name, read straight off the request, so a virtual key sent there survived the filter. The new module-private_credentialless_caller_key_valuesreadsgeneral_settings(read-only, via the same lazy import already used elsewhere in the module) and returns the set of accepted key values; the filter now value-strips any header matching one of them. Blast radius unchanged: both the helper and the filter are called only from the single existing caller, no new raises (the 401 gate is untouched), no mutation, no signature change. Verified live on both sides: at the merge-base withlitellm_key_header_name: x-company-keyset,x-company-key: Bearer <vkey>forwarded toaiplatform.googleapis.comalongside a real Google key; on this tip the same request forwards only the real Google key with the custom header and the virtual key stripped, and the custom header alone returns 401. Cases A-F re-verified unchanged on this tip./live-pr-risk CHECKED ab93636: SAFE. This commit replaces the hand-rolled name-drop set with one derived from the canonical
SpecialHeaders.litellm_credential_header_names()minus the two headers that double as real Google credentials (Authorization,x-goog-api-key), which are value-stripped instead. This is the same sourceuser_api_key_authreads the caller's key from, so the drop set now cannot drift out of sync with what authenticates, and it picked upOcp-Apim-Subscription-Key, which the hand-rolled set missed.SpecialHeadersis a pure enum already imported into the module via its_typesstar import; the derived frozenset is evaluated once at import with no side effects, and both module-level constants are referenced only inside the single-caller filter, so the blast radius is unchanged. Verified live on both sides: at the merge-base a distinct caller Azure APIM secret inOcp-Apim-Subscription-Keyforwarded toaiplatform.googleapis.com; on this tip it is dropped while the genuine GoogleX-Goog-User-Projectheader is preserved, and all of Cases A-G re-verified unchanged./live-pr-risk CHECKED f3dc339: SAFE. This commit resolves the caller key by the same precedence
get_api_keyuses (customlitellm_key_header_name, thenx-litellm-api-key,Authorization,api-key,x-api-key,x-goog-api-key,Ocp-Apim-Subscription-Key) and value-strips exactly the one value that authenticated, instead of only the value fromx-litellm-api-key/Authorization/ the custom header. This closes the structural gap Greptile flagged:x-goog-api-keyis an accepted auth source that the old caller-key set omitted while keeping the header, so a virtual key authenticated through it would have been forwarded. The precedence tuple is built once at import from the sameSpecialHeadersenum;_authenticated_caller_key_valuesreads request headers andgeneral_settingsread-only and is still called only by the single-caller filter, so the blast radius is unchanged, no new raises, no mutation. Verified live at this tip: a real Google key inx-goog-api-keyalongside a higher-precedence virtual key inx-litellm-api-keyis still forwarded (Case E, 599), the bring-your-own OAuth token is still forwarded (Case C, 599), and a virtual key sent only inx-goog-api-keyis rejected. Note the proxy's own auth currently rejects a virtual key presented inx-goog-api-keyon this route before the filter runs, so this commit is defense-in-depth on the forwarding boundary, proven directly by the added unit tests./live-pr-risk CHECKED 5d8286c: SAFE. This commit swaps the filter's own
Bearer-only stripping for the auth module's_get_bearer_token, so the caller-key comparison normalizes exactly the schemes authentication accepts (Bearer/bearer/Basic/AWS4-HMAC-SHA256), with a raw-value fallback for a bare token. It closes the case Greptile flagged: a virtual key echoed asAuthorization: Basic <key>alongside a higher-precedence auth header did not match the caller key under the old normalization and was forwarded._get_bearer_tokenis a pure function inuser_api_key_auth(already imported into this module foruser_api_key_auth), with no side effects; the new_normalize_credential_valuewrapper is referenced only by the single-caller resolver and filter, so the blast radius is unchanged. Verified live on both sides: at the merge-baseAuthorization: Basic <vkey>forwarded toaiplatform.googleapis.com; on this tip it returns 401 with nothing forwarded, while the bring-your-own OAuth token (Case C) and a real Google key (Case E) still forward with the virtual key absent./live-pr-risk CHECKED 2fe1e7e: SAFE.
user_api_key_authalso accepts the caller key from apass_through_endpointsentry'sheaders.litellm_user_api_key, not onlylitellm_key_header_name. This commit adds_operator_configured_caller_key_header_names, which reads both fromgeneral_settings(read-only), drops every configured caller-key header by name, and feeds them as top-precedence caller-key sources into the resolver. The helper is pure and referenced only by the single-caller resolver and filter, so the blast radius is unchanged, no new raises, no mutation. Config values are read defensively withisinstanceguards. Covered by unit tests that configure each source throughgeneral_settingsand assert the configured header is dropped while a real Google key inx-goog-api-keyis preserved; the full passthrough test file (160 tests) passes, including the LIT-4761 streaming-classification suite whose fixture now sends the virtual key inx-litellm-api-key, matching a real request./live-pr-risk CHECKED fcc047b: SAFE. Corrects the precedence of the operator-configured key headers to match
get_api_keyexactly:litellm_key_header_nameoverrides everything so it resolves first, then the built-in headers inget_api_keyorder, then apass_through_endpointslitellm_user_api_keyheader whichget_api_keychecks last. The prior commit had lumped both configured sources at the top, so a request that authenticated viaAuthorizationwhile also carrying a pass-through header could pick the wrong value and leave the authenticatedAuthorizationkey forwarded._operator_configured_caller_key_header_namesnow returns(override, pass_through)and both the resolver ordering and the name-drop consume it; still pure, still called only by the single-caller resolver and filter, no new raises or mutation. Covered by a new unit test whereAuthorizationholds the authenticated key, a pass-through header holds a decoy, and a realx-goog-api-keyis present: theAuthorizationkey is stripped, the pass-through header dropped, and the real Google key preserved. Full passthrough test file (161 tests) green./live-pr-risk CHECKED 16a81c9: SAFE. Closes a high-severity vector Cursor Bugbot flagged: on mapped pass-through routes (of which
/vertex_aiis one),user_api_key_authaccepts the caller key from a header literally namedlitellm_user_api_keyviacheck_api_key_for_custom_headers_or_pass_through_endpoints, applied last so it overrides every other source. The filter now drops that header by name and resolves it at highest precedence. Adds a module constant plus a prepend to the resolver order and a union into the name-drop set; still pure, still called only by the single-caller resolver and filter, no new raises or mutation. Verified live on both sides: at the previous tip a virtual key inlitellm_user_api_keyforwarded toaiplatform.googleapis.comwhile the realAuthorizationwas wrongly stripped; on this tip the virtual key is dropped and both the bring-your-own OAuth token and a real Google key are preserved. Full passthrough test file (163 tests) green.Note
High Risk
Security-sensitive change to Vertex passthrough auth and header forwarding; wrong filtering could break BYO-Google flows or still leak secrets upstream.
Overview
Fixes LIT-5997: when the proxy has no Vertex credential, the bring-your-own-credentials branch no longer forwards the full incoming header set to Google.
Credential-less Vertex passthrough now builds upstream headers via
_forwarded_headers_for_credentialless_vertex_passthroughinstead of copying all request headers. Proxy-only auth headers (fromSpecialHeaders.litellm_credential_header_names()exceptAuthorization/x-goog-api-key, plus operatorlitellm_key_header_nameand pass-through key headers) are dropped by name. The caller key is resolved with the same precedence asuser_api_key_authand stripped by value from any remaining header (using_get_bearer_tokenfor scheme normalization). If neither a survivingAuthorizationnorx-goog-api-keyremains, the route returns 401 with guidance instead of calling Google.Bring-your-own Google OAuth or API keys still forward; LiteLLM virtual keys in
Authorization,x-litellm-api-key,x-goog-api-key, or custom headers no longer reach upstream. Tests were updated and expanded (TestVertexCredentiallessPassthroughVirtualKeyLeak) for these cases.Reviewed by Cursor Bugbot for commit 16a81c9. Bugbot is set up for automated code reviews on this repo. Configure here.